Fast estimation of fractal dimension and correlation integral on stream data

Given a cloud of N points in an E-dimensional space, we often need to estimate the intrinsic dimensionality D of this cloud. For example, a set of points in 3-dimensional space all following along a straight line has intrinsic (or fractal) dimensionality D=1. Non-integer fractal dimensionality appears pervasively in nature. In this paper we give a very fast method to estimate the fractal dimensionality of the points in a data stream. Algorithms to estimate the fractal dimension exist, from the straightforward quadratic algorithm, to the faster O(NlogN) or even O(N) algorithms that use the so-called box-counting method. However, these algorithms require Ω(N) space, and hence are ill-suited to semi-infinite streams of data. In this paper we propose an algorithm, based on a “tug-of-war” idea, which computes the fractal dimension in a single pass over the dataset using only constant memory. Experimental results on synthetic and real world data sets demonstrate the effectiveness of our algorithm.